Skip to content

perf(queue): coalesce the four event-driven job types by their per-invocation id - #2001

Merged
JSONbored merged 2 commits into
mainfrom
claude/coalesce-event-jobs
Jul 1, 2026
Merged

perf(queue): coalesce the four event-driven job types by their per-invocation id#2001
JSONbored merged 2 commits into
mainfrom
claude/coalesce-event-jobs

Conversation

@JSONbored

@JSONbored JSONbored commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Summary

The four event-driven job types — run-agent, notify-deliver, notify-evaluate, submit-draft — had no coalesce key, so a duplicate enqueue (a webhook redelivery, a retried dispatch) queued a second identical job. Every one of them already carries a stable per-invocation id, so this adds coalesce keys off those:

type id used
run-agent runId
notify-deliver deliveryId
notify-evaluate event.dedupKey
submit-draft draftId

Because the key is the per-invocation id, coalescing only ever merges a true duplicate re-enqueue of the same job — two distinct invocations have distinct ids and therefore distinct keys, so nothing legitimate is ever dropped. A payload missing its id returns null (uncoalesced) rather than sharing a key.

This is the rec #17 half of the #1942 audit's coalesce cluster. Advances #1936.

Notes on the other two recs in that cluster (kept out of this PR):

  • rec feat(agent): make next-action recommendations repo-specific #16 (repair-data-fidelity skip already-pending): the maintenance jobs it enqueues (backfill-*, generate-signal-snapshots, …) already have coalesce keys, so a re-enqueue this tick already coalesces — the "skip" is effectively in place.
  • rec fix(docs): use Gittensor screenshot in homepage hero #11 (extend coalesce to status IN (pending, processing)): deliberately deferred — coalescing onto an in-flight job is materially riskier (it can swallow an update that must re-run after the current pass), so it deserves its own PR + a recoalesce marker rather than riding along here.

Validation

  • git diff --check, npm run typecheck
  • npm run test:coverage — new test: each type coalesces by its id, distinct ids stay distinct, and a missing id → null (covers both branches of every new case); existing coalesce/priority suites still pass.
  • npm run test:ci, npm audit --audit-level=moderate

Safety

  • No secrets/etc. Pure queue-key logic.
  • Cannot drop legitimate work: the key is the unique per-invocation id; a missing id is left uncoalesced.

JSONbored added 2 commits July 1, 2026 00:11
@JSONbored
perf(review): early-exit the re-review when a swept PR is already clo…
ae47338 
…sed on GitHub

reReviewStoredPullRequest only checked the STORED state, then spent GitHub budget
resyncing files + running readiness/CI reads + re-reviewing even for a PR already
closed/merged on GitHub (a dropped `closed` webhook while the relay was down).

After the single memoized fetchLivePullRequest, early-exit on a live NON-open
state: upsert the live payload to reconcile the stored row, then return before the
files + CI reads + review. Reconciling the row lets the next sweep skip it via the
existing stored-state guard. Fail-open: only a live non-open state early-exits.

Advances #1936.
@JSONbored
perf(queue): coalesce the four event-driven job types by their per-in…
97e9ee0 
…vocation id

run-agent / notify-deliver / notify-evaluate / submit-draft had no coalesce key,
so a duplicate enqueue (webhook redelivery, retried dispatch) queued a second
identical job. Each carries a stable per-invocation id (runId / deliveryId /
event.dedupKey / draftId), so key off that: coalescing then only ever merges a
true duplicate re-enqueue of the SAME job — distinct invocations have distinct
ids (distinct keys), and a missing id stays uncoalesced (null), so nothing
legitimate is dropped.

Advances #1936.
@dosubot dosubot Bot added the size:M label Jul 1, 2026
@loopover-orb

loopover-orb Bot commented Jul 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-01 07:25:23 UTC

4 files · 1 AI reviewer · no blockers · readiness 95/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review

Review summary
The diff adds coalesce keys for the four event-driven job types using their per-invocation identifiers and returns null when the identifier is absent, so malformed payloads stay uncoalesced instead of sharing a broad key. It also adds a separate terminal early-exit in `reReviewStoredPullRequest` that reconciles a live non-open PR before doing file/CI reads, and the added test covers that path. The queue-key change is narrow and the visible logic preserves distinct invocations as long as those id fields are truly unique at their producers.

Nits — 5 non-blocking
  • nit: `src/queue/processors.ts:1813` is an unrelated terminal-regate behavior change in a PR whose title and summary describe only event-job coalescing, so split it out or explicitly document it in the PR description.
  • nit: `test/unit/selfhost-queue-common.test.ts:543` bundles all four new coalesce cases and all missing-id cases into one test, which makes failures less precise than one table-driven assertion per job type.
  • `src/queue/processors.ts:1813`: either move the terminal early-exit and `#regate-terminal-exit` test to a separate PR or update this PR's summary/safety section to explain that second behavior change.
  • `test/unit/selfhost-queue-common.test.ts:543`: make the coalesce expectations table-driven with columns for `type`, id field, expected key, and missing-id payload so future event-driven job types are harder to forget.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:M; no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 68 registered-repo PR(s), 58 merged, 209 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 68 PR(s), 209 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 68 PR(s), 209 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Triage stale or unlinked PRs.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 1, 2026
@JSONbored JSONbored self-assigned this Jul 1, 2026
@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.66%. Comparing base (be006e8) to head (97e9ee0).
⚠️ Report is 10 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2001      +/-   ##
==========================================
- Coverage   95.66%   95.66%   -0.01%     
==========================================
  Files         220      220              
  Lines       24453    24465      +12     
  Branches     8868     8874       +6     
==========================================
+ Hits        23394    23404      +10     
  Misses        436      436              
- Partials      623      625       +2
Files with missing lines Coverage Δ
src/queue/processors.ts 89.75% <100.00%> (-0.11%) ⬇️
src/selfhost/queue-common.ts 91.94% <100.00%> (+0.27%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit f54b057 into main Jul 1, 2026
13 checks passed
@JSONbored
JSONbored deleted the claude/coalesce-event-jobs branch July 1, 2026 07:28
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jul 1, 2026
@JSONbored JSONbored mentioned this pull request Jul 1, 2026
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@JSONbored JSONbored

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

No open projects
gittensory - v1 roadmap
Status: Done

Milestone

M3.5 — Agent Layer Phase 2 (maintaine...

Development

Successfully merging this pull request may close these issues.

1 participant

@JSONbored